cpi uiaii 



Resolving social dilemmas on evolving random networks 



Attila Szolnoki 1 and Matjaz Perc 2 



ON 
O 
O 

(N 
-i— > 
o 

O 

o 



43 

6 
o 

O 
Oh 



> 

in 
o 

On 



O 



X 



1 Research Institute for Technical Physics and Materials Science, P.O. Box 49, H-1525 Budapest, Hungary 

2 Faculty of Natural Sciences and Mathematics, University of Maribor, Koroska cesta 160, SI-2000 Maribor, Slovenia 



PACS 02 . 50 . Le - Decision theory and game theory 
PACS 87 . 23 . Ge - Dynamics of social systems 

PACS 89 . 75 . Fb - Structures and organization in complex systems 

Abstract. - We show that strategy independent adaptations of random interaction networks 
can induce powerful mechanisms, ranging from the Red Queen to group selection, that promote 
cooperation in evolutionary social dilemmas. These two mechanisms emerge spontaneously as 
dynamical processes due to deletions and additions of links, which are performed whenever players 
adopt new strategies and after a certain number of game iterations, respectively. The potency 
of cooperation promotion, as well as the mechanism responsible for it, can thereby be tuned 
via a single parameter determining the frequency of link additions. We thus demonstrate that 
coevolving random networks may evoke an appropriate mechanism for each social dilemma, such 
that cooperation prevails even by highly unfavorable conditions. 



Introduction. — Social dilemmas constitute situa- 
tions where the collective wellbeing is at odds with indi- 
vidual success. Frequently studied within the framework 
of evolutionary game theory [1] , they provide some of the 
most challenging environments for the sustenance of coop- 
erative behavior. As successors of evolutionary games on 
regular grids [2-4], evolutionary games on complex net- 
works have been the subject of intense investigations in 
the last years [5], and their ability to promote or sustain 
cooperative behavior in different types of social dilemmas 
has often been confirmed [6-14]. Apart from some no- 
table early exceptions [15, 16] the majority of these stud- 
1 ies considered static interaction networks underlying the 
main evolutionary process. The evolution on networks is, 
however, increasingly often accompanied also by the evo- 
lution of networks [17-26], and this not just in the context 
of evolutionary game theory. Indeed, networks are to be 
seen as evolving or adaptive entities that may substan- 
tially influence any dynamical process that is taking place 
on them [27]. 

Coevolutionary processes in general, i.e. processes that 
happen alongside the main evolution of strategies, are cur- 
rently in the focus of attention within evolutionary game 
theory, and it has been shown that they may very effec- 
tively promote cooperation (see e.g. [22,25,26]). Notably, 
the subject of coevolution need not be the interaction net- 
work, but the term may refer to the teaching activity and 
related reproduction capability [28], or the ability of the 



players to move on the spatial grid [29-31], as well. An im- 
portant observation in many cases is that coevolutionary 
rules may lead to highly heterogeneous states in a spon- 
taneous manner [32]. Since heterogeneity has often been 
found favorable for the evolution of cooperation [33,34], 
it is considered a key outcome of coevolutionary processes 
that lead to enhanced levels of cooperation. 

Here we aim to show that simple coevolutionary rules 
affecting the interaction network may lead, not just to 
heterogeneous states promoting the cooperative strategy, 
but also to new dynamical processes that positively af- 
fect the evolution of cooperation. In particular, we start 
with a random interaction network and introduce a co- 
evolutionary rule entailing both deletions of existing and 
additions of new links between players. While existing 
links are deleted whenever a player adopts a new strat- 
egy or its degree exceeds a threshold value, new links are 
added after each given number r of game iterations. The 
latter parameter thus defines a time scale for the addi- 
tion of new links, which may be tuned faster or slower 
according to the deletion of existing links. Irrespective 
of the time scale separation [35, 36] between them, the 
counteraction of deletions and additions of links largely 
preserves the initial random topology of the network and 
its heterogeneity, so that the reported ability of resolv- 
ing social dilemmas is due to the spontaneous emergence 
of the Red Queen mechanism and group selection, which 
appear spontaneously in dependence on r and the social 
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dilemma governing the evolution of the strategies. We 
thus show that simple strategy independent coevolution- 
ary rules may evoke appropriate dynamical mechanisms 
that affect the adoption of strategies on the macroscopic 
level of evolutionary games so that the governing social 
dilemma is resolved in favor of the collective wellbeing. 

The remainder of this letter is organized as follows. 
First, we describe the considered social dilemmas and the 
protocol for the coevolution of random networks. Next 
we present the results, whereas lastly we summarize and 
discuss their implications. 

Social dilemmas and setup. — In what follows we 
consider all three major social dilemma types where play- 
ers can choose either to cooperate or defect, whereby mu- 
tual cooperation yields the reward R, mutual defection 
leads to punishment P, and the mixed choice gives the co- 
operator the sucker's payoff S and the defector the temp- 
tation T. Adopting previously introduced parametriza- 
tion [12], thus designating R = 1 and P = as fixed, 
the remaining two payoffs can occupy — 1 < S < 1 and 
< T < 2, where if T > R > P > S we have the 
prisoner's dilemma game, T > R > S > P yields the 
snowdrift and R > T > P > S the stag-hunt game. Ir- 
respective of the governing social dilemma, initially each 
player x is designated either as a cooperator (s x = C) or 
defector (s x = D) with equal probability, and is placed 
on a random network that is constructed from N indi- 
viduals with an average degree k avg — A. As usual, du- 
plicate links are omitted. Evolution of the two strategies 
is performed in accordance with the Monte Carlo simula- 
tion procedure comprising the following elementary steps. 
First, a randomly selected player x acquires its payoff p x 
by playing the game with all its k x neighbors. Next, one 
randomly chosen neighbor of x, denoted by y, also ac- 
quires its payoff p y by playing the game with all its k y 
neighbors. Last, if p x > p y player x tries to enforce its 
strategy s x on player y in accordance with the probability 
W(s x — > s y ) = (p x ~p y )/bkq, where k q is the largest of 
the two degrees k x and k y , as used before in case of het- 
erogeneous interaction topologies [9,12,22]. In accordance 
with the random sequential update, each player is selected 
once on average during a full Monte Carlo step. 

In addition to the evolution of the two strategies by each 
considered social dilemma, a rule for the adaptation of ran- 
dom interaction networks is implemented. First, whenever 
player x adopts a new strategy all its links, except from 
the one with the donor of the new strategy, are deleted. 
This process of strategy adoption and simultaneous link 
deletion is demonstrated schematically in Fig.[TJ Hence, in 
addition to adopting a new strategy, the player is forced to 
break its connections to former allies and getting k x = 1. 
Second, to counteract the depletion of links that consti- 
tute the random network, all individuals are allowed to 
form a new link with a randomly chosen player with which 
they are not yet connected after every r full Monte Carlo 
steps. The latter process can be considered as aging, and 




Fig. 1: Schematic presentation of the coevolutionary rule af- 
fecting the interaction network. Player depicted by the black 
circle passes strategy to its neighbor depicted by the white 
circle. Consequently, the invaded player looses all its links, ex- 
cept from the one with the donor of the new strategy. This is 
marked by the dashed lines extending towards other neighbor- 
ing players depicted by grey circles. 

accordingly, as soon as k x reaches a threshold k max , player 
x dies and is replaced by a newborn having the same strat- 
egy and keeping a single randomly selected link from its 
predecessor, thus maintaining k x — 1. Within the current 
work k max was chosen large enough so as not to influence 
the initial random topology of players. Below presented 
results were obtained on networks hosting N = 10 4 — 2-10 5 
players, for which k max — 500 has proven to be sufficiently 
large. It is worth mentioning that the coevolutionary pro- 
cess affecting the interaction network may occasionally re- 
sult in detached individual players that originally formed 
the neighborhood of an invaded player. In such cases we 
relinked the detached player randomly back onto the net- 
work with a single connection. By maintaining the mini- 
mal degree equal to one, we achieve that every player has 
at least one neighbor at all times, which is a necessary 
condition for playing the game and defining the fitness of 
an individual. We also highlight that the proposed rule 
for the adaptation of interaction networks is fully strat- 
egy independent, and thus, on its own does not indirectly 
support cooperators by treating C — C, C — D or D — D 
links differently from one another. Next, we will system- 
atically analyze the evolution of cooperation by r = 1 
and t = 500 for all three social dilemma types. The two 
considered values correspond to small and large time scale 
separation between link deletions and additions, and as we 
will show, largely affect the sustenance of the cooperative 
strategy as well as the mechanisms responsible for it. 

Results. — We start by presenting results obtained 
for t = 1 in Fig. [2j where the complete T — S phase di- 
agram is presented. In the bottom right quadrant, where 
the T > R > P > S ranking constitutes the prisoner's 
dilemma game, the cooperators can be sustained up to 
T = 1.1 but only if S = 0, whereas for all S < -0.037 the 
defectors dominate completely. Moving to the stag-hunt 
game in the bottom left quadrant (R > T > P > S) 
of Fig. O the sustenance of cooperators improves, yet 
in both cases cooperation levels do not notably surpass 
those that can be obtained on random networks in the 
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Fig. 2: Full T — S phase diagram obtained for r = 1. Solid 
red lines denote the borders between full cooperator dominance 
(C phase), full defector dominance (D phase), and the mixed 
states (C + D phase). Dotted blue line in the snowdrift quad- 
rant (upper right) denotes the border of oscillatory solutions 
(O phase). We further examine this quadrant along the dash- 
dotted black line using the common r parametrization (see text 
for details) . Dashed green lines depict borders between the so- 
cial dilemma types. Note that the dominance of cooperators 
in the upper left quadrant is trivial since T < 1 and S > 
simultaneously. 

absence of coevolutionary rules (see e.g. Fig. 2 in [12]). 
Indeed, the most interesting features can be observed in 
the upper right quadrant, where besides complete cooper- 
ator dominance for low enough T, there exist a broad re- 
gion of mixed C + D states including oscillatory solutions 
(marked with O). Evidently, the dynamical adaptations 
of the random network can sustain cooperators across the 
whole snowdrift quadrant, which is a notable improvement 
if compared to the case of static interaction networks. 

Aiming to identify the mechanism behind the promo- 
tion of cooperation, we analyze the evolutionary processes 
in the upper right quadrant of Fig. [3] more accurately. 
Without loosing generality we can introduce a single pa- 
rameter r G (0, 1) that determines the free payoff elements 
as T = 1 + r and S = 1 — r [37] . Accordingly, the values of 
r, constituting the dash-dotted diagonal in Fig. [2j charac- 
terize the cost-to-benefit ratio of the snowdrift game [9]. 
In Fig. [3] we present the density of cooperators pc along 
r. Since the r diagonal cuts through the oscillatory so- 
lutions (dotted blue region in Fig. we plot in Fig. [3] 
minima and maxima of pc for these particular values of r. 
Following complete cooperator dominance up to r = 0.41 
and a rather sharp descent of pc (region around arrow a) , 
the oscillatory solutions start via a second order continu- 
ous phase transition at r = 0.50. The amplitudes of these 
oscillations are therefore initially modest (arrow b), but 
increase fast beyond the transition point (arrow c). As 
r increases further the oscillations terminate abruptly via 
a first order discontinuous phase transition at r = 0.732 
(arrow d), and again settle onto a stationary state. Tem- 



o 

Q_ 




r 



Fig. 3: Density of cooperators pc in dependence on r (the dash- 
dotted diagonal splitting the upper right quadrant in Fig. [2} for 
t — 1. Arrows mark sampling values of r for which temporal 
curses of pc are presented in Fig. [4] Note that by oscillatory 
solutions, starting at r — 0.50 (arrow b) via a second order 
continuous phase transition and ending at r — 0.732 (arrow 
d) via a first order discontinuous phase transition, both the 
minima and maxima of pc are depicted. The vertical dashed 
connecting lines at r = 0.732 indicate that these states are 
unstable and transient either to the oscillatory or the steady 
state solution [see Fig. |4jd) and the pertaining main text] . 

poral behaviors of pc at these characteristic values of r 
are depicted in Fig. [4] 

From the dynamical point of view particularly notable 
is Fig. fjjd) , where the coexistence of steady state and os- 
cillatory solutions, characteristic for some first order dis- 
continuous phase transitions, is depicted. We note here 
that the exact sequence of bifurcations is difficult to de- 
termine in the absence of a low-dimensional model or the 
use of coarse-graining techniques (see [38] for a related 
study). The basin of attraction of the steady state so- 
lution is caught if the initial value of pc is close enough 
to the stationary value (initially pc = 0.6; dashed blue), 
but otherwise the oscillatory solution is chosen (initially 
pc = 0.2; solid red). From the viewpoint of strategy evo- 
lution, results presented in Figs. [3] and |4] outline a Red 
Queen mechanism for the sustenance of cooperation in 
the snowdrift game, similarly as has been reported previ- 
ously if a third strategy, such as loners, was introduced 
to evolutionary games (see e.g. [39,40]). Since oscillations 
are possible only if the system has at least two indepen- 
dent variables, presently the obvious candidate for the sec- 
ond, first being pc, is the network structure. Indeed, by 
defining pik as the density of players whose degree belongs 
to the lower class (e.g. k < (k max /3)) and measuring it 
in dependence on time, it can be verified that it changes 
with the same oscillation frequency as pc but a 7r/2 phase 
shift, as shown in Fig. [5l We thus argue that the tides of 
cooperation are evoked by the emergence of robust coop- 
erative clusters that can form around players having high 
degree, similarly as reported previously for scale-free net- 
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Fig. 4: Temporal evolutions of pc for r corresponding to the 
arrows in Fig. U (a) r = 0.42, (b) r = 0.50, (c) r = 0.64 and 
(d) r = 0.732. In panel (d) the coexistence of a steady state 
(dashed blue line) and the oscillatory solution (solid red line) at 
the first order discontinuous phase transition is demonstrated 
(see text for details). 



works [9]. However, since according to the revolutionary 
rule the links of these players are dissolved as soon as 
they reach k max , the cooperative clusters disintegrate, in 
turn enabling the defectors to win and thus resulting in 
the downfall of pc- Simultaneously, the density of players 
having small(large) degree rises(falls), as evidenced by the 
dashed blue line in Fig. The process starts anew when 
frequent additions of new links due to a low r value tem- 
porarily restore the difference between influential players 
having degree close to k max and the followers who have 
comparatively low degree, thus inducing a new oscillation 
cycle. 

Remarkably, when the time scale separation between 
link deletions and additions increases, the strength of co- 
operation promotion also increases substantially, as evi- 
denced in Fig. [5] obtained for r = 500. Note that the 
complete dominance of cooperators [pc — 1) now spans 
over an extensive T — S region, encompassing the entire 
traditional snowdrift (0 < r < 1; see e.g. [9]) and weak 
prisoner's dilemma (1 < T < 2, R=l, P=S=0; see e.g. [2]) 
parametrization. Moreover, the stag hunt dilemma is re- 
solved very satisfactory as well, with defectors dominating 
only by extremely harsh conditions that are characterized 
with high T and extensively negative S < —0.6. If com- 
pared to results presented in Fig.[2](T = 1), the improve- 
ment in terms of cooperation promotion in Fig. [6] is obvi- 
ous, and it can also be observed that the mixed (C + D) 
and oscillatory (O) phases vanish. We thus conclude that 
the mechanism responsible for the promotion of coopera- 
tion changes as well, which may be attributed to two cru- 
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Fig. 5: Temporal evolutions of pc (solid red line) and the den- 
sity of players belonging to the low-degree class pit (dashed 
blue line) obtained for r — 0.7 (equivalently T = 1.7 and 
S — 0.3) and r = 1. Note the anti-phase synchronization 
between pc and the fraction of players having small degree. 



cial difference compared to the r = 1 case. First, due to 
the slower addition of new links the cooperative domains 
can grow larger and stronger around players having high 
degree since they prevail over longer periods of time, i.e. 
the k ma ,x threshold is not reached so fast. And second, 
the slow additions of new links facilitate the emergence 
of highly influential players, with comparatively very high 
degrees if compared to the majority, to which the follow- 
ers cannot catch up easily. These two facts result in a 
transition from the Red Queen mechanism of cooperation 
promotion by the snowdrift game and the predominantly 
heterogeneity based promotion of cooperation in the pris- 
oner's dilemma and the stag hunt game at r = 1, to a 
powerful group selection mechanism that emerges by all 
three social dilemmas for higher t, as we will demonstrate 
next. 

Figure [7J shows temporal courses of pc from the snow- 
drift quadrant at r = 0.85 for increasing values of r. Most 
importantly, we point out the emergence of time intervals 
during which pc is constant. This cascade-like feature be- 
comes increasingly pronounced as r increases. Namely, 
at t = 1 (solid red line) and r = 50 (dashed green line) 
it is practically absent, whereas at r = 500 (dash-dotted 
black line) the cumulative duration of dormancy of pc sur- 
passes that of active phases. We argue that the reason for 
the emergence of these time intervals of inactivity lies in 
the introduced coevolutionary process, which if r is suf- 
ficiently large, meaning that new links are added slowly, 
leads to the emergence of homogeneous and virtually iso- 
lated groups of players. These groups remain inactive for 
as long as it takes for the newly added links to recon- 
nect them with one another, of which duration is roughly 
equivalent to t (see Fig. [7]). It is important to note that 
during the inactive phase there are practically no strat- 
egy transfers taking place, and thus the main source of 
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Fig. 6: Full T — S phase diagram obtained for r = 500. 
Solid red line denotes the sharp border between full cooperator 
dominance (C phase) and full defector dominance (D phase). 
Dashed green lines depict borders between the social dilemma 
types. 

link deletions is disabled. Consequently, the addition of 
new links can gradually reconnect the detached groups, 
which then again triggers an avalanche of strategy adop- 
tions, which in turn starts the whole process again, until 
an absorbing state is reached. Thus, the temporal plots 
in Fig. [7] arguably evidence the spontaneous emergence of 
group selection due to the introduction of the proposed 
coevolutionary rule by high values of r. Moreover, since 
the slow additions of new links facilitate the emergence of 
highly influential players, the cooperators flourish within 
the isolated groups whereas defectors weaken. Note that 
influential defectors are exposed to a negative feedback 
effect that sets in as soon as the neighbors adopt the de- 
fecting strategy. Then there is nobody left to exploit, and 
such clusters become vulnerable to cooperators. Thus, as 
soon as the two types of groups reestablish a sufficiently 
strong interconnectedness, cooperators can successfully in- 
vade the defectors, thereby gradually increasing the coop- 
erative domains. The latter processes manifest as rather 
steep jumps in the temporal traces of pc by large enough 
r, which are then again followed by dormancy since the 
many strategy adoptions anew lead to isolation of homo- 
geneous groups of players and restart the outlined group 
selection mechanism. 

Summary. — In sum, we show that evolving random 
networks constitute an optimal environment for the evolu- 
tion of cooperation provided the time scale separation be- 
tween the deletions and additions of links is large enough. 
A powerful group selection mechanism then emerges spon- 
taneously, which is able to warrant full cooperator domi- 
nance across an extensive T — S region covering all major 
types of social dilemmas. However, if the additions of 
new links are frequent, the formation of isolated homoge- 
neous groups is hindered and the sustenance of coopera- 
tors relies either on the heterogeneity of random interac- 



Fig. 7: Temporal evolution of pc at r = 0.85 (equivalently 
T = 1.85 and S = 0.15) for r = 1 (solid red line), r = 50 
(dashed green line), r = 200 (dotted blue line) and r = 500 
(dash-dotted black line) . Steps in the temporal evolution (time 
intervals during which pc is constant) that can be observed by 
larger values of r indicate the emergence of group selection in 
the snowdrift game. Qualitatively identical results can also be 
obtained for the other two considered social dilemma types. 

tion networks, or as demonstrated for the snowdrift game, 
on a Red Queen mechanism which emerges due to the 
interplay between the oscillatory changes of the network 
structure and the density of cooperators. The complexity 
of this process is underlined by the first order discontin- 
uous phase transition responsible for the termination of 
the oscillatory phase that postulates coexistence of steady 
state and oscillatory solutions which are chosen in depen- 
dence on the initial conditions, thus indicating the option 
of bistability in coevolutionary games. Notably, the exis- 
tence of multiple equilibria has recently been reported also 
for iV-person stag hunt dilemmas [41], whereas oscillatory 
solutions in the context of evolutionary games have been 
presented in [42]. Simple strategy independent coevolu- 
tionary rules, entailing both additions of new as well as 
deletions of existing links, can thus favor cooperative be- 
havior beyond the levels of static complex networks, and 
seem to offer a rich plethora of mechanisms to tackle this 
formidable challenge. Thereby processes such as making 
new friends, punishment or aging, which may all account 
for the adaptations of interaction networks, are an integral 
part of everyday life, and it therefore seems natural to in- 
corporate them into the framework of evolutionary game 
theory in order to aid the evolution of cooperation. 
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